Skip to content

feat: add anonymous Scarf usage telemetry (FB-1354)#59

Merged
dig-whois merged 5 commits into
mainfrom
fb-1354-scarf-telemetry-v2
Jul 20, 2026
Merged

feat: add anonymous Scarf usage telemetry (FB-1354)#59
dig-whois merged 5 commits into
mainfrom
fb-1354-scarf-telemetry-v2

Conversation

@dig-whois

@dig-whois dig-whois commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • send one privacy-minimized aggregate Scarf event per day from the elected operator leader
  • route public operator, default engine, and Helm chart pulls through Firebolt-owned Scarf gateways
  • make telemetry.enabled=false disable runtime events and switch default operator/engine image pulls to GHCR
  • preserve every user-supplied operator, FireboltEngine, and FireboltEngineClass image repository

Privacy

The event includes operator/engine versions, Kubernetes minor, OS/architecture, and bucketed instance/engine/replica counts. It contains no names, stable identifiers, query data, schemas, secrets, or configuration. The source IP is visible to Scarf for company inference but is not stored.

The README pixel is disclosed separately because it operates independently of runtime settings.

Opt-out behavior

telemetry.enabled=false provides one control for runtime telemetry and default image routing. To bypass Scarf for the chart download itself, install from oci://ghcr.io/firebolt-db/helm-charts; Helm selects the chart URL before reading values. DO_NOT_TRACK and SCARF_NO_ANALYTICS disable runtime events only.

Managed deployment dependency

Firebolt-managed deployments must land their DO_NOT_TRACK=1 follow-up PRs before this chart is released. Managed compute charts already override operator and engine repositories to the internal ECR mirror.

Test plan

  • make test
  • make lint
  • make docs-check
  • make helm-docs
  • helm lint helm/firebolt-operator
  • verify default render uses Scarf repositories
  • verify telemetry.enabled=false renders GHCR repositories and --telemetry=false
  • verify a custom operator repository remains unchanged when telemetry is disabled

Note

Medium Risk
Changes default image/chart pull paths and adds outbound telemetry; misconfiguration could surprise air-gapped installs, though opt-out and GHCR bypass are documented and user image repos are preserved.

Overview
Introduces anonymous Scarf usage telemetry: the elected operator leader runs a new internal/telemetry reporter that sends at most one aggregate event per day (bucketed instance/engine/replica counts, versions, K8s minor, OS/arch) via --telemetry / --telemetry-endpoint, with DO_NOT_TRACK / SCARF_NO_ANALYTICS respected through scarf-go.

Default distribution shifts to Firebolt’s Scarf gateways: Helm chart and docs use oci://oci.firebolt.io/..., default operator and embedded engine images use oci.firebolt.io, and README adds Scarf badges plus a disclosed README pixel. telemetry.enabled=false (or --telemetry=false) disables runtime events, passes that flag from the chart, and when the chart’s default operator repo is unchanged, rewrites the operator image to GHCR; the binary calls UseDirectEngineRepository() so the operator’s fallback engine repo becomes GHCR while CR/engine/class image overrides stay intact.

Local dev and CI are updated (make local-deploy disables telemetry, Kind mirrors oci.firebolt.io). Privacy and opt-out behavior are documented in README, installation, and quickstart.

Reviewed by Cursor Bugbot for commit d0fc3b8. Bugbot is set up for automated code reviews on this repo. Configure here.

@dig-whois
dig-whois requested a review from a team as a code owner July 14, 2026 11:02
@fireboltdb-docs-integration

fireboltdb-docs-integration Bot commented Jul 14, 2026

Copy link
Copy Markdown

Mintlify docs preview

Docs were aggregated into the Firebolt documentation site.

Status Preview Updated (UTC)
🟢 Ready View Preview Jul 20, 2026, 9:27 AM

Mintlify may take a few minutes to finish building. This comment updates on each sync while the source PR is open.

gm42
gm42 previously approved these changes Jul 14, 2026
Report one privacy-minimized aggregate event per day from the elected operator leader, with transparent Helm controls and documentation so public adoption is measurable and users can opt out.
@dig-whois
dig-whois force-pushed the fb-1354-scarf-telemetry-v2 branch from c6b6690 to 6c50f3e Compare July 20, 2026 07:01
Comment thread internal/telemetry/telemetry.go
Comment thread internal/telemetry/telemetry.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Not approving: Cursor Bugbot found 2 unresolved issues (including a medium-severity daily-report guarantee concern) and its check skipped; Security Agent passed. Assigned fstr and leonard-firebolt for follow-up.

Open in Web View Automation 

Sent by Cursor Approval Agent: Infra Reviewer

@cursor
cursor Bot requested review from fstr and leonard-firebolt July 20, 2026 07:04
Use the Firebolt-owned gateway for customer-facing operator and default engine images so artifact adoption is measured alongside chart downloads.
Comment thread config/images/defaults.latest.env
Comment thread README.md

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: high. Not approving: Cursor Bugbot skipped with 3 unresolved findings (including high-severity Kind mirror breakage) and Security Agent reported a medium-severity README tracking-pixel issue. fstr and leonard-firebolt are already assigned for follow-up.

Open in Web View Automation 

Sent by Cursor Approval Agent: Infra Reviewer

Delay the first event by a full interval to preserve the daily guarantee, reuse the standard image parser, mirror the Scarf registry in Kind, and disclose README pixel behavior.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 00df367. Configure here.

Comment thread config/images/defaults.latest.env

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: high. Not approving: Cursor Bugbot skipped with 1 unresolved high-severity finding (default engine registry forces rollouts); Security Agent passed. fstr and leonard-firebolt are already assigned for follow-up.

Open in Web View Automation 

Sent by Cursor Approval Agent: Infra Reviewer

When telemetry is disabled, bypass Scarf for the default operator and engine images while preserving every user-supplied image repository.
cursor[bot]
cursor Bot previously approved these changes Jul 20, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Approved: Cursor Bugbot and Security Agent passed on the latest commit with no new findings, and prior automated review threads are resolved. fstr and leonard-firebolt remain assigned for human review.

Open in Web View Automation 

Sent by Cursor Approval Agent: Infra Reviewer

Keep configuration comments focused on durable behavior rather than internal telemetry rationale.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: medium. Approved: Cursor Bugbot and Security Agent passed on the latest commit with no new findings, and all prior automated review threads are resolved. fstr and leonard-firebolt remain assigned for human review.

Open in Web View Automation 

Sent by Cursor Approval Agent: Infra Reviewer

@dig-whois
dig-whois enabled auto-merge (rebase) July 20, 2026 09:31
@dig-whois
dig-whois disabled auto-merge July 20, 2026 09:46
@dig-whois
dig-whois enabled auto-merge (squash) July 20, 2026 09:48
@dig-whois
dig-whois disabled auto-merge July 20, 2026 09:48
@dig-whois
dig-whois enabled auto-merge (rebase) July 20, 2026 09:48
@dig-whois
dig-whois disabled auto-merge July 20, 2026 10:08
@dig-whois
dig-whois merged commit 90368e4 into main Jul 20, 2026
32 checks passed
@dig-whois
dig-whois deleted the fb-1354-scarf-telemetry-v2 branch July 20, 2026 12:10
@fireboltdb-docs-integration

Copy link
Copy Markdown

Packdb docs aggregation ready for review

Docs were merged to main and aggregated into packdb.

Squash-merge the packdb PR into master to publish the docs. Urgent fixes can be cherry-picked to the active release/packdb-* branch like any other docs change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants